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Abstract 

The work on the estimation of the condition numbers of Vandermonde matrices, motivated by appli¬ 
cations to interpolation and quadrature, can be traced back at least to the 1970s. Empirical study has 
shown consistently that Vandermonde matrices tend to be badly ill-conditioned, with a narrow class of 
notable exceptions, such as the matrices of the discrete Fourier transform (hereafter referred to as DFT). 

So far formal support for this empirical observation, however, has been limited to the matrices defined by 
the real set of knots. We prove that, more generally, any Vandermonde matrix of a large size is badly ill- 
conditioned unless its knots are more or less equally spaced on or about the circle (7(0,1) = {* : |a:| = 1}. 

The matrices of DFT are perfectly conditioned, being defined by a cyclic sequence of knots, equally spaced 
on that circle, but we prove that even a slight modification of the knots into the so called quasi-cyclic 
sequence on this circle defines badly ill-conditioned Vandermonde matrices. Likewise we prove that the 
half-size leading block of a large DFT matrix is badly ill-conditioned. (This result was motivated by an 
application to pre-conditioning of an input matrix for Gaussian elimination with no pivoting.) Our analy¬ 
sis involves the Ekkart-Young theorem, the Vandermonde-to-Cauchy transformation of matrix structure, 
our new inversion formula for a Cauchy matrix, and low-rank approximation of its large submatrices. 

Keywords: Vandermonde matrices; Condition number; Cauchy matrices; Cauchy Inversion formulae; Van¬ 
dermonde inverse 
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1 Introduction 

Our main subject, motivated by applications to interpolation and quadrature, is the estimation of the 
condition number, k(V s ), of a nonsingular n x n Vandermonde matrix, V s , where 

K(Us) = ||U s || HOI, V s = (si)Zj} o, (1.1) 

s = denotes the vector of n distinct knots so, ■ • •, s„_i, and \\M\\ = ||M|| 2 denotes the spectral norm 

of a matrix M. First of all, one would like to know whether this number is reasonably bounded or large, 
that is, whether the matrix is well- or ill-conditioned. 

It has been proven in [ GI881 . [G90] , and lT94j that the condition number k(T 4) is exponential in n if all 
knots So, Si,..., s n -1 are real. This means that such a matrix is badly ill-conditioned already for moderately 
large integers n, but empirically almost all Vandermonde matrices of large or even reasonably large sizes are 
badly ill-conditioned as well, with a narrow class of exceptions, such as the Vandermonde matrices defined 
by cyclic sequences of knots, equally spaced on the circle U(0,1) = {x : \x\ = 1}. Such matrices are unitary 
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up to scaling by 1 /y/n and thus are perfectly conditioned, that is, have the minimum condition number 1 . 
A celebrated example is the matrix of the discrete Fourier transform (DFT), 

ft = for u} q = exp(27r\/^T/g) (1.2) 

denoting a primitive qth root of 1. 

A slight modification of the knots into the so called quasi-cyclic sequence can make the matrices badly 
ill-conditioned, however. This has been an empirical observation of IG901 Section IV], whose formal support 
turned out to be elusive. The critical step was the estimation of the norm of the inverse matrix, ||V)T 1 ||. 
The known expressions for the entries of this matrix are not explicit enough. In particular, write 

n —1 n —1 

s{x) = = s(x)/((x - Si)s)(si)) = ^2 Sijx J , (1.3) 

1=0 i,j=0 

and recall (G90l equation (2.3)] that 


v- 


n— 1 


= ( s bl)"i=o> and so ||V s || > max \ Si>j \ 

,J z,.7=0 


(1.4) 


(see other expressions in )M031 , [SEMCll i, and the references therein). As Walter Gautschi wrote in IG90 . 
Section IV], the known lower and upper bounds on the condition number of a Vandermonde matrix, ” unfor¬ 
tunately, are too far apart to give much useful information “, and so he resorted to “computing the condition 
numbers... numerically 11 . 

Our new lower bounds enable us to support formally the cited empirical observations for a very general 
class of Vandermonde matrices. Our results (see Sections [4] and |8j) indicate that the condition number of an 
n x n Vandermonde matrix is exponential in n unless its knots are more or less equally spaced on or about 
the unit circle C( 0,1). 

Our study covers the matrices defined with the quasi-cyclic sequence of knots as a special case (see Section 
0, and we further observe in Section 0 that all k x k blocks of the n x n matrix of discrete Fourier transform, 
for n = 2k, are badly ill-conditioned as long as the integers k and n are sufficiently large, even though the 
n x n DFT matrix itself is unitary up to scaling. 

In order to prove our lower bounds on the condition number k(I 4), we shift from Vandermonde matrices 
v s of dm) to Cauchy matrices 


' S j, t n ' % 


n— 1 


,j=0 


(1.5) 


defined by 2 n distinct knots so, • • •, s n - 1 ; to,, t n -i- The matrices of these two classes are much different, 
and it is not obvious that their studies can be linked to one another. Unlike a Vandermonde matrix 14, a 
Cauchy matrix (7 Sj t has rational entries, its every submatrix is a Cauchy matrix as well, and the Cauchy struc¬ 
ture is invariant in column interchange as well as in scaling and shifting all entries by the same scalar, that 
is, aC aSta t = C S! t for a ^ 0 and Cs+ae.t+ae = C Sj t for e = (1,..., 1) T . Moreover, some powerful technique 
has been developed in the papers [MRT05) , |R06| , |P15) , [Paj , |XXG12| , [XXCB14] for the approximation of 
reasonably large submatrices of Cauchy matrices by lower-rank matrices. 

Nevertheless there is a simple Vandermonde-Cauchy link, which has become the basis for our progress 
in this paper as well as in ESI and (Paj . (The paper P 89/90] has revealed a more general link among 
the matrix structures of Vandermonde, Cauchy, Toeplitz and Hankel types, based on their displacement 
representation. The paper has also pointed out potential algorithmic applications, and this work has indeed 
become the springboard for devising the highly efficient algorithms of the subsequent papers iGK()95| , [G981 , 
[MEM!, ESS], IXXC12I . ixxcbi4| . IpIKI . and [Paj l.1 


Restatement of our original task in terms of Cauchy matrices has open new opportunities for our study 
of Vandermonde matrices, and we obtained the desired progress by extending the known formula for the 
Cauchy determinant to the following simple expression for all entries of the inverse of a Cauchy matrix C Si t, 


i C Zt) 




(_l)n 

- s(tj')t(si>) for s(x) = JJ(x—Sj) of (11.31) . t(x) = (x — tj), and all pairs (■ i',j' ). (1.6) 

V l ' i— o i=o 
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We applied this expression in order to prove exponential lower bounds on the norms of the inverses and 
the condition numbers of Cauchy matrices of a large class, and we extended these bounds to Vandermonde 
matrices. We have also deduced alternative exponential lower bounds by applying the known techniques for 
low-rank approximation of certain large submatrices of Cauchy matrices. 

In Section 0 by applying the Ekkart-Young theorem, we more readily prove the exponential growth of 
the condition number of the Vandermonde matrices of a more narrow class, namely, the ones having at least 
a single knot outside the circle (7(0,1) or having many knots inside this circle. These results provide simple 
initial insight into the subject. 

Otherwise we organize our presentation as follows. In the next section we recall some basic definitions 
and auxiliary results. In Section [4] at first we deduce our basic lower bounds on the condition number n(V s ) 
by applying a link between Vandermonde and Cauchy matrices; then we informally interpret these bounds 
in terms of some restrictions on the knot set {so, Si,..., s„_i} of any well-conditioned Vandermonde matrix. 
In Sections [5] and [G] we prove exponential growth of the condition number of Vandermonde matrices, defined 
by the quasi-cyclic sequence of knots and the half-size leading blocks of the DFT matrices, respectively. In 
Section [7] we deduce an exponential lower bound on the condition number of the Cauchy matrices closely 
linked to Vandermonde matrices. In Section [8] we extend this bound to an alternative exponential lower 
bound on the condition number k(V s ). This bound is weaker than the bounds of Section |4l for the matrix 
classes of Sections [5] and [6l but its domain of applications is shown more explicitly. In Section [9] we present 
the results of our numerical tests. Section [TO] is left for conclusions. 

2 Some Definitions and Auxiliary Results 

&j(M) denotes the jth largest singular value of an n x n matrix M. 

\\M\\ = ||M ||2 = &i(M) is its spectral norm. 

k(M) = ui(M)/a p (M) is its condition number provided that p = rank(M). 

<t p (M) = ||M _1 || and n(M) = \\M\\ ||M _1 || if p = n , that is, if M is a nonsingular matrix. 

Theorem 2.1. (The Ekkart-Young Theorem.) is equal to the distance ||M — between M and 

its closest approximation by a matrix Mj_\ of rank at most j — 1. 

Corollary 2.1. Suppose that a matrix B has been obtained by appending k new rows or k new columns to 
a matrix A. Then crj(A) > crj+k(B) for all j. 

Fact 2.1. For the DFT matrix 12 and u> n of \1.2\) . it holds that 

(i) 111211 = ||12 ff || = \fn and 

(ii) nO- 1 =Q H = (u;W')™“i 0 . 

3 The Condition of Vandermonde Matrices: Simple Bounds 

For a nonsingular Vandermonde matrix 14 of write 

s + = max|sj|, |I4| = max{l,s” -1 }. (3.1) 

i =0 

Next we prove that the condition number k(V s ) is exponential in n if s+ > v > 1 (see Theorem 13.11 part 
(i)) or in k if l/|sj| > v > 1 for at least k knots s;, i = ii,... ,ik (see Corollary 13.II) . 

Theorem 3.1. For a Vandermonde matrix 14 = ( s i)r7=0’ ^ holds that 
(i) |I4| < ||I4|| < n 1141 and (ii) ||I4|| > y/n. 

Proof. The theorem readily follows from [GL131 equations (2.3.8) and (2.3.11)]. □ 

Theorem 3.2. For a Vandermonde matrix 14 = q, it holds that 

(i) cr n (I4) < y/n and 

(ii) a n (V s ) < n 1 ~ k y/km.ax{k, vj(i / — 1)} if l/|sj| > v > 1 for i = 0,1,..., k — 1, k > 1. 
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Proof. At first turn the matrix V s into a rank deficient matrix by setting to 0 the entries of its first column. 
Then note that the norm of this perturbation equals yfn and deduce part (i) by applying Theorem 12.11 
Likewise, turn the matrix 14 into a rank deficient matrix by setting to 0 its entries that lie in its first k rows, 
but not in its first k — 1 columns. 

This perturbation, diag(s* _1 )|r 0 1 (s^) i , has column norm, || • ||i, at most i/ 1 ~ k k and has row norm, 

|| • | |oo, at most p 1 ~ k Y^i=k vkl — 1PT- So its s P ec tral norm is at most u 1 ~ k \/k rnaxjfc, v/(v — 1)}. Hence 
Theorem 12.II implies part (ii). □ 

Corollary 3.1. For a Vandermonde matrix 14 = (sj)™J_ 0 , 

(i) k(V s ) > max{l,s" _1 /- v /n} (cf. Table \9.1\) and 

(ii) k( 14) > |14|^ fc_1 /(V , /cmax{fc, v j(v — 1)}) if l/|sj| > v > 1 for i = 0,1,..., k — 1 (cf. Table HO)) . 

Proof. Combine Theorems 13.11 and 13.21 with equation (13.11) . □ 

Remark 3.1. Observe that ||14||oo = (s+ — l)/(s + — 1) for s+ 1, and so ||14|| > (s™ — l)/((s + — 1 )y/n) 
(cf. \GLltA equation (2.3.7)]). This refines slightly equation \3.1\) and hence Corollarv \3. 1\ 

Corollarv l3.1l shows that an n x n Vandermonde matrix 14 is badly ill-conditioned already for a moderately 
large integer n if s+ > v or if l/|si| > u, for a constant v > 1, i = 0, 1,..., k— 1, and a reasonably large integer 
k. Therefore a Vandermonde matrix 14 of a large size is badly ill-conditioned unless all knots so,..., s„_i 
lie in or near the disc {x : \x < 11} and unless they lie mostly on or near its boundary cirle C(0,1). In the 
following we prove stronger restrictions on the knots of a well-conditioned Vandermonde matrix. 


4 The Condition Estimates via the Inversion Formulae 

Let Mij denote the (i, j)th minor of a nonsingular nxn matrix M, that is, the determinant of its submatrix, 
obtained by deleting its ith row and jth column of the matrix M. Then 

M~ l = (-l) I+J (M i y/det(M)) ij . (4.1) 

Next we estimate the norm of the inverse of a nonsingular Cauchy matrix C s .t of USD, by combining 
equation (sm with the following well known formula, 

det(C s ,t) = — s i)(ti ~ tj)/ — tj)- (4-2) 

i<j i,j 

Since every square submatrix of a nonsingular Cauchy matrix is again a nonsingular Cauchy matrix, we 
can extend equations (14.21) to its minors, combine these extensions with equation (14.21) itself and with (14.11) . 
and deduce (after some simplifications) that 


C 1 y n +i +j ^ ^ ^ ^ ^ r ^ ^ 

(Cat)*',? = , _ tw) _ nw -o/( n ^ n ^~ ^x^ -^ 

' 1 3 ' i j i<j' i'<j 

for every pair (i',j'). In particular, note that n 2 + n — 1 is an odd number and obtain 


)) (4-3) 


(Ot )n-r,o = 


1 


(to — Sn-l) 




(-ir 


(to — Sn- 1) 


^(to^tfSn.— 1 ), 


(4.4) 


for the polynomials s(x ) = — s i) an d t(x) = n”=o ( x ~ C of ULCD - By re-enumerating the knots S; 

and tj , we can turn any pair of them into the pair of s n _i and to and thus arrive at expression m- 

We are going to combine this expression with the following equation (cf. |P15| equation (5)]), which links 
Vandermonde and Cauchy matrices, 


V s = DC s . t Df l V t and V~ x = Vf 1 D- 1 . 


(4.5) 
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Here 


(4.6) 


D = diag(t(s i ))r=o 1 > D i = diag(t'(tj))” =0 \ 

for the polynomial t{x) = n"=o 1 ( :E — t%) °f <11.611 . 

Equations (14.41) (14.61) combined can be considered inversion formulae for a nonsingular Vandermonde 
matrix 14 where to,..., t n -i are n parameters of our choice. Let us express these parameters and consequently 
the inverse matrix V" 1 through a single complex nonzero parameter /. Write tj = /wj), for j = 0,..., n — 1, 
and thus define the subclass of the CV matrices, C s j = (-—1— r)7j=o^ i n t ^ ie c l ass of Cauchy matrices C s . t . 

Substitute C Sj t = C s j, 14 = V/ = fidiag(/■ 7 )"r 0 1 , and t(x ) = x n — /" into equation (14.51) . and obtain 

V s = D f C s jD^ f n diagff)- 1 , 


for 


D f = diag(s" - DS, 1 and D 1J = ndiagK/u*)"- 1 )^ 1 . 
Combine these equations with Fact mu substitute ^ = uj u ^ for all j, and obtain 


V- 1 = diag(.T- 1 -^ 1 ^ diag)JJT q 1 C~j diag (- 


- /"/i=o 


(4.7) 


where the matrix C s ^ is defined by equation (14.31) for tj = fui J n and j = 0 ,... ,n— 1. 

Equations m , m , and tj = for |/| = 1 and j = 0, ...,n — 1, combined can be considered 
inversion formulae for a nonsingular Vandermonde matrix 14 depending on the single nonzero parameter /. 

Furthermore we can substitute t(x) = x n — /" into equation (14.41) and express any entry (Cf t ‘ ),< iJ ' of the 
matrix as follows, 

(QjW = (-l)MV)(s”' - /")/(*' - tj.). (4.8) 

Equations (14.71) and (14.81) . combined together for all pairs i' and j', provide a simplified representation of 
the inverses of CV and Vandermonde matrices. Next we apply this representation in order to estimate the 
condition number of a Vandermonde matrix. 


Theorem 4.1. Let V s be a nonsingular nxn Vandermonde matrix of EUl. Fix a scalar f such that \ f\ = 1 
and Si ^ fu>° n for all pairs of i and j, thus implying that the CV matrix C s j is well defined and nonsingular. 
Then k(V s ) > sfh \\C~j 11/max^ 1 1 s? - f n \. 

Proof. Apply equation (14.71) and note that the matrix diag(/ Tl_1_ - ? )" ; r 0 1 H ff diag(w4- 7 ) 7 V 0 1 is unitary. There- 

( \ n— 1 

-przjx) II- Hence ||14 _1 || > HC^/H/max'Vg |s™ — f n |. Combine this estimate 
with the bound n(V s ) > ||K _1 |lv / ”' of Theorem 13.II □ 

Corollary 4.1. Let 14 be a nonsingular n x n Vandermonde matrix. Then 


k(V s ) > \fn max |s(/)|/ max |sj — f \ for six ) of (11.61) . 

|/|=1 0<i<n-l 

Proof. An infinitesimal perturbation of the parameter / can ensure that the matrix C s j is well defined, and 
so we can apply Theorem l4.ll Substitute (14.81) for tj> = f and for i! that maximizes the value |s" — f n \. □ 

Corollary 4.2. Assume a nonsingular nxn Vandermonde matrix 14 with < 1 for s+ of Corollaru \3.1l 
Write Vi = s(cu^), i = 0,..., n, and v = (vi)£— 0 . Then 
k{V s ) > y/n max|/| =1 |s(/)|/2 > ||v||/2. 

Proof. Note that |sj — f \ < 2 for |sj| < s + < 1 and that max|/| =1 |s(/)| > HvH^ > ||v||/-y/n. □ 
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Corollary 14.21 suggests that a Vandermonde matrix 14 of a large size is badly ill-conditioned if the knot 
set {so,..., s n — i} is sparse in some disc that covers a reasonably thick neighborhood of a reasonably long 
arc of the circle C(0,1). Thus, informally, the knots of a well-conditioned Vandermonde matrix must be 
more or less equally spaced on or about the circle C(0,1). 

Towards a more formal support of the latter informal claim, one should estimate the growth of the 
coefficients of the polynomial s(a;) when the knots s, deviate from such a distribution. We estimate this 
growth in some special cases in the next two sections. Then, based on part (i) of Theorem 14.11 and low-rank 
approximation of CV matrices, we prove rather general results of this kind, although only for the inputs of 
large sizes. 

We conclude this section by estimating the condition number k(V s ) in terms of the coefficients of the 
polynomial s(x) rather than its values. 

Corollary 4.3. Under the assumptions of Corollary \).2\ write s(x) = x n + Y^i -0 fi x% j /n = 1, and f = 

(/*)?=, o- ThenK(V s )>0.5\\i\\V^+l- 

Proof. Note that v = fl„ +1 f for the vector v of Corollary 14.21 and fi n+ i = (w ra+ 1 )" =0 . Therefore f = 
fl4+i v = ^r^^+i v - Consequently ||v|| = ||f||-\/n + 1 because Ll]f +1 is a unitary matrix. Substitute 
this equation into Corollary 14.21 □ 

For a fixed set {so,..., s n _i}, one can compute the coefficients <tq, • • •, cr„_i of s(x) by using 0(nlog 2 (n)) 
arithmetic operations (cf. .EQU Section 2.4]), and then bound the condition number k(V s ) by applying 
Corollary IQ 

5 The Case of the Quasi-Cyclic Sequence of Knots 

If all the knots so,..., s n -± 

(a) lie on the unit circle C(0,1) and 

(b) are equally spaced on it, 

then we arrive at perfectly conditioned Vandermonde matrices, such as the DFT matrix f2, satisfying n(fl) = 
1. A rather minor deviations from assumption (b), however, can make the matrix ill-conditioned, as [G90] 
has shown empirically and we deduce formally from Theorem 14.II next. 

Example 5.1. The quasi-cyclic sequence of knots. 

For applications to interpolation and quadrature, one seeks a sequence of nxn well conditioned Vander¬ 
monde matrices for n = 2, 3,..., such that all the knots Si are reused recursively as n increases (cf. (C90\ 
Section IV]). For n = 2 k , we choose the nxn matrix of DFT at n points, defined by the equally spaced knots 
Si = exp(2ni-\/^-l/2 k ) for i = 0,1,..., 2 fc — 1. We can keep these knots for all n > 2 k as well, but then how 
should we choose the remaining knots Si if n is not a power of 2? A rather straightforward choice, called the 
quasi-cyclic sequence in ] G90\ Section IV], is the sequence Si = exp(27r/)\/— 1), i = 0,1,..., n — 1, defined 
by the following sequence of fractions fi, 

f| 1131357J__3__5_J7_Jli!i3I5 

u ’ 2 > 4 > 4 ’ 8 > 8 > 8 > 8 ’ 16 ’ 16 ’ 16 > 16 ’ 16 ’ 16 ’ 16 ’ 16 ’ ' ' ' 

In other words, one can use the DFT matrix for n = 2 k and positive integers k, and then, as n increases, can 
extend the set so,..., s 2 fe_i of the 2 k -th roots of 1 recursively by adding, one by one, the remaining 2 k+l -st 
roots of 1 in counter-clockwise order. For n = 2 k , these are the unitary matrices of DFT, but for n = 3 • 2 k 
and k > 4, their condition numbers become large, exceeding 1,000,000 already for n = 48 (cf. f G90\ Section 
IV]). This has been remaining an empirical observation since 1990, but next we deduce from Corollary \f.2\ 
that log(ft(T4)) grows at least proportionally to 2 k , that is, k(V s ) grows exponentially in n. 

Theorem 5.1. Let V q denote the nxn Vandermonde matrix defined by the quasi-cyclic sequence of knots 
on the unit circle C(0,1) for n = 3 q, q = 2 fc_1 , and a positive integer k. Then n(V q ) > 2 q / 2 y/n. 

Proof. For n = 3 q, the sequence has the 2 q knots s* = w 2g , i = 0,1,..., 2q — 1, equally spaced on the 
circle C(0,1), and the q additional knots s, = w|* +1 , i = 0, 1 ,..., q — 1 , equally spaced on the upper half 
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of that circle such that s(x) = ( x 2q — 1 )p q (x), for p q (x) = — w 4 g +1 )- Corollary 14.21 implies that 

K,(V q ) > \s(f)\y/n/2, where we can choose any complex / such that |/| = 1. 

Choose / = — W 4 q and then observe that f 2q — 1 = — 2, and so in this case 

|s(/)| = 2|p g (/)|. (5.1) 

Furthermore, \ f — > y/2, for i = 1,..., q. Consequently, |s(/)| > 2 ? / 2+1 , because 

K(Vq)>0.5\s(f)\V^, (5.2) 

and the theorem follows. □ 

The theorem already implies that the condition number k(I 4) grows exponentially in n for n = 3q and 

q = 2 fc_1 , but we can strengthen the estimate a little further because of the following observation. 

Fact 5.1. |/ — > 2 cos((0.5 — i/q)ir/2) for q being a power of two and i in the range between 0 and q. 

This fact implies that the distance \f — of^ rl \ grows from \[2 to 2 as i grows from 0 to q/2, but this 
distance decreases back to y/2 as i grows from q/2 to q. In the limit, as q —> oo, we obtain 

\p q (f) I > exp (y) for y = / ln(2cos((0.5 - x/q)ir/2)) dx. (5.3) 

Jo 

For large integers q this expression closely approximates lower bounds, which we can obtain for any q by 
replacing the integral with a partial sum. Let us do this for q divisible by 12. Replace the integral with a 
partial sum by applying Fact 15. II for i = q/ 6 and 2cos((0.5 — i/q)n/2) = y/3 and obtain 

\p q (f )I > 18 9/6 . (5.4) 

Refine this estimate by applying Fact 15.II for i = q/ 3 and 2cos((0.5 — i/q) tt/2) ss 1.9318517 and deduce that 

\p g (f )| > (2cos(7r/12)v / 6) 9/3 . (5.5) 

For n = 48 the lower bound of Theorem 15. II on n{V q ) is just 2 s \/48 = 1024-\/3 > 1, 773.6, but (15.41) and 
(1531) enable us to increase this lower bound to 15,417 and 27,598, respectively. By engaging Fact 15. II at first 
for i = q/ 4 and 2cos((0.5 — i/q)n/2) ~ 1.9615706 and then for i = q/ 12 and 2cos((0.5 — i/q)Tr/2) ~ 1.586707, 
we increase these lower bounds at first to 38,453 and then to 71,174. We leave further refinement of the 
bounds as an exercise for the reader. 

Can one still define a sequence of well-conditioned h x h Vandermonde matrices for h = 1,... ,n, which 
would include all the 2 fc th roots of 1 for k = [log 2 n\ and would consists of only n knots overall? IG90[ 
Section 4] proves that such a sequence of well-conditioned h x h Vandermonde matrices is obtained if we 
define the knots Sj = 2 irfiy/—l, i = 0,l,...,n — 1, by the following van der Corput sequence of fractions fi, 
which reorders the knots of quasi-cyclic sequence in a zigzag manner, 

nll31537J^_9__5_13_3_U_7_15 
u ’ 2 ’ 4 ’ 4 ’ 8 ’ 8 ’ 8 ’ 8 ’ 16 ’ 16 > 16’ 16 ’ 16 ’ 16 > 16’ 16’"" 

The knots Si,..., s n are equally spaced on the unit circle C'(0,1) for n = 2 k being the powers of 2 and 
are distributed on it quite evenly for all n. Gautschi has proved the following estimate in [G901 Section 4], 

Theorem 5.2. Given the van der Corput sequence of n x n Vandermonde matrices 14 for n = 1,2,3,..., 
it holds that k(V s ) < \[2sn for all n. 

6 Numerical Instability of Gaussian Elimination with No Pivoting 
for the Matrices of Discrete Fourier Transform 

Gaussian elimination with partial pivoting, that is, with appropriate row interchange, is extensively used in 
matrix computations, even though pivoting is a substantial burden in the context of the modern computer 
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technology. The papers |PQY15| and [PZ151 list the following problems with pivoting: it interrupts the 
stream of arithmetic operations with foreign operations of comparison, involves book-keeping, compromises 
data locality, complicates parallelization of the computations, and increases communication overhead and 
data dependence. The user prefers to apply Gaussian elimination with no pivoting (hereafter we use the 
acronym GENP ) unless it fails or runs into numerical problems, which occurs if and only if some leading 
blocks of the input matrix A are singular or ill-conditioned (cf. |PQZ13[ Theorem 5.1]). 

These observations have motivated substantial interest to numerical application of GENP and the efforts 
towards supporting it with various heuristic and randomization techniques (see |PQY15| and |BCD14! for 
history accounts). Our next theorem contributes to the study of this important subject. 

Theorem 6.1. Assume for convenience that n is even and let denote the q x q leading (that is, north¬ 
western) block of the matrix f l for q = n/ 2. Then > 2 rl / 4 - 1 - v /n. 

Proof. As follows from Corollary 14.21 7 c(fi (9 )) > \s(fco l q )\^/n/2 where s(x ) = YljZl(x — <*;£) and where we 
can choose / = — yf— 1. Under such a choice ]/ — uj° n \ > v/2 for j = 0,.. ., q — 1, and so |s(/)| > 2 n / 4 . Hence 
Ac(f > 2 ra / 4 -yn/2, and Theorem 16. II follows. □ 

The estimates of equations (15.21) (15.51) still hold if we substitute p q (x) = s(x) and n = 2 q, which should 
replace equations (15.11) and n = 3q of the previous section, and so we can strengthen the bound of Theorem 
16.11 accordingly. 

The techniques of the proof of Theorem 16.11 can be extended to reveal that all the k x k blocks of the 
unitary matrix are badly ill-conditioned as long as the integers k or n — k are large. This observation 
can lead to interesting conclusion about randomized structured preprocessing of GENP, studied in |PQZ13| , 
|PQY15| , IP Z 151 . and |PZaj . Namely, together with |PQY15[ Theorem 6.2] and [PZ15I Theorem 9] Theorem 
16.11 implies that Gaussian random circulant multipliers support numerically stable application of GENP to 
the matrix fl only with a probability near 0, even though they have consistently supported numerical GENP 
in the extensive tests with matrices of various other classes in |PQZ13| and |PQY15| . 

Finally, in some contrast to Theorem 16.11 one can turn all leading blocks of the matrix Q into well- 
conditioned matrices, by re-ordering its rows according to the van der Corput sequence. 

Theorem 6.2. Given the van der Corput sequence of n x n Vandermonde matrices 14 for n = 1,2,3,, 
the leading block of any of the matrices in the sequence is also a matrix in the same sequence. 

Combine this observation with Theorem l5.2l and conclude that all leading blocks of a Vandermonde matrix 
defined by a van der Corput sequence of knots are nonsingular and well-conditioned, and so application of 
GENP to such a Vandermonde matrix is numerically stable. 


7 Low-Rank Approximation of Cauchy and CV Matrices 

Next, under some mild assumptions on the knots So,..., s m _i, we deduce an exponential low bound on the 
norm of the inverse of a nonsingular CV matrix C s j , which is the reciprocal of its smallest singular value, 
11(77)11 = l/cr n (C 8 j). At first we deduce such a bound for a submatrix and then extend it to the matrix 
itself by applying Corollary 12. II In the next section, by extending this estimate, we prove our alternative 
exponential lower bound on the norm ||I 4 ~ 1 |I an( l the condition number k( 14 ) of matrix 14 , which holds if 
the knot sets of the matrix 14 deviates substantially from the evenly spaced distribution on the circle (7(0,1). 

Definition 7.1. (See \CGS0T\ page 125f].) 

Two complex points s and t are (rj, c) -separated, for 77 > 1 and a complex center c, if | < I/ 77 . 

Two complex sets S and T are (q, c)-separated if every pair of points s £ S and t G T is ( 77 , c)-separated. 

Theorem 7.1. (Cf. \CGS0i\ Section 2.2], •PIT Theorem 29].) Letm and l denote a pair of positive integers 
and let S m = {so,..., s m _i} and Ti = {to, ..., t/_ 1 } denote two sets that are ( 77 , c)-separated for a pair of 
real 77 > 1 and complex c. Define the Cauchy matrix C = ( s .l t . )™~=o’* Write 6 = <5 c ,s = minU ^ 1 \si — c|. 
Then 


l/a p (C) > (1 — rf)q p x 5 for all positive integers p. 



































We apply the theorem to any knot set S m and to some specific knot sets 7). 

Theorem 7.2. Let us be given a complex f such that \ f\ = 1 and two knot sets S m = {so> ■ • • ,s m -i} and 
Tn = {toj • • • > tn- 1 } where tj = fuifi for j = 0,..., n — 1, ui = exp(27ri/--T/n). Define the Cauchy matrix 

C m ,n ( si—tj )*G"STn,ie7k • 

Fix two integers j' and j" in the range [0, n — 1] such that 0 < l = j" — j' + 1 < nj 2 and define the subset 
of the set T n - 

Let c = 0.5 (tp + tj") denote the midpoint of the line interval with the endpoints tj> and tj" and let 
r = |c — tj' | = |c — tj" | = sin^'" — j')^) denote the distance from this midpoint to the points tj' and tj". 
(Note that ( j" — j')^ is a close upper bound on r for large integers n.) 

Fix a constant 77 > 1 and partition the set S m into the two subsets S mj - and of cardinalities m_ and 
m+, respectively, such thatm = to_+to+, the knots of S m - He in the open disc D(c,rjr) = {z : \z—c\ < r/r}, 
and the knots of S m ,+ He in the exterior of that disc. 

(i) Then the two sets S m ,+ and Tj’,j" are (c,rf)~separated and 

(ii) 1 /a p+ m^+n-i(Cm, n ) > g p (rj - 1 )r for all positive integers p. 

Proof. Part (i) follows because distance(c, Tj'j") = r and because distance(c, S mt +) > pr by virtue of the 
definition of the set 

Now define the two following submatrices of the matrix C m ,n , 


C„ 


(s,- — tj) 


i&SmJ&T,, 


and C, 


m+J ,J 



i€S m ,+,jeT } ij" 


Apply Theorem 17.II to the matrix C = Cm+,j',j" and 5 = pr and obtain that 


1 /cT p (C m+ jpj") > r] p (r] — 1 )r for all positive integers p. 


Combine this bound with Corollary 12.11 applied for k = m_ and obtain that 


1 /a p + m _(C rn p'j") > rj p (r] — 1 )r for all positive integers p. 

Combine the latter bound with Corollary [2J] applied for k = n — l and obtain part (ii) of the theorem. □ 

Corollary 7.1. Under the assumptions of Theorem \ 7. let m = n and p = l — ni- > 0. Then C s j = C m ^ n 
and 


IICsjII = l/vn{C a ,f) > t ? p (?7 - l)r. 


8 A Typical Large Vandermonde Matrix is Badly Ill-conditioned 

Combine part (i) of Theorem 14. II and Corollary 17. II and obtain the following result. 

Corollary 8 . 1 . Under the assumptions of Corollary \7.1[ let s + < 1 for s+ of equation \S.1\) . Then 

K(V a ) > V p (ri — l)r\fn/2. 

The corollary shows that, under its mild assumptions, the condition number k( 14) of an n x n Vander¬ 
monde matrix 14 grows exponentially in p , and thus also in n if p grows proportionally to n. 

To specify an implication, fix any real rj > 1 and any pair of integers j' and j" in the range [0, n — 1] such 
that 0 < j" — f < n/2. Then the disc D(c , r) contains precisely l = j" — j’ + 1 knots from the set T, and 
m_ denotes the number of knots of the set S in the disc D(c,r/r). Hence Corollary 18.II implies that n(V s ) is 
exponential in n unless l — m_ = o(n), for all such triples with l = j" — j' + 1 of order n , that is, 

unless the number of knots Si in the disc -D(c, rjr ) exceeds, matches or nearly matches the number of knots 
tj in the disc D(c,r) for all such triples. Clearly, this property is only satisfied for all such triples (jq,j',j") 
if the associated knot set S is more or less evenly spaced on or about the unit circle C(0,1). 


9 





9 Numerical Tests 


Numerical tests have been performed by Liang Zhao in the Graduate Center of the City University of New 
York. He has run them on a Dell server by using Windows system and MATLAB R2014a (with the IEEE 
standard double precision). 

Table [9J] displays the condition numbers n(V s ) of n x n Vandermonde matrices, with the knots s, = uj l n 
being the nth roots of 1, for * = 0,1,..., n — 2, and with the knot s n _i in the range from 1.14 to 10. For 
comparison the table also displays the lower estimates of part (i) of Corollary 13.II in the case of m = n. By 
setting s n -i = we would have k(V s ) = 1, but with the growth of the value |s n _i| and even with the 

growth of n for s n _i fixed at 1.14, the matrix was quickly becoming ill-conditioned. 

Table 19.21 displays the condition numbers ofnxn Vandermonde matrices defined by various pairs of 
integers k and n and the knots so,...,s n -i such that S; = u> l n _ k are the (n — fc)th roots of 1, for i = 
0,1,..., TZ- — k — 1, and s n _ k+ i = p<7 are the fcth roots of 1 scaled by p, for * = 0,1,..., k and p = 3/4,1/2. 
For comparison the table also displays the lower estimates «_ = |I4| l/fc_1 /(v / fc max{fc, v/{v — 1)}) of part 
(ii) of Corollary 13.11 for v = 1/p. 


Table 9.1: Condition Numbers of Vandermonde Matrices with a Single Absolutely Large Knot 


11 

Sn— 1 

k(V„) 

si 7Vn 

64 

1.14E+00 

3.36E+03 

4.98E+02 

64 

1.56E+00 

6.88E+11 

2.03E+11 

64 

3.25E+00 

4.62E+32 

2.22E+31 

64 

1.00E+01 

5.47E+63 

1.25E+62 

128 

1.14E+00 

1.08E+07 

1.60E+06 

128 

1.56E+00 

3.24E+32 

3.64E+23 

128 

3.25E+00 

6.40E+76 

9.03E+63 

128 

1.00E+01 

1.30E+130 

8.84E+125 

256 

1.14E+00 

1.57E+14 

2.33E+13 

256 

1.56E+00 

6.61E+62 

1.66E+48 

256 

3.25E+00 

2.89E+132 

2.12E+129 

256 

1.00E+01 

1.18E+265 

6.25E+253 


Table 9.2: Condition Numbers of Vandermonde Matrices with k Absolutely Small Knots 




CO 

II 

P= 1/2 

n 

k 

k(V s ),k- 

k{V s ),k- 

64 

8 

4.04E+01, 7.14E+00 

6.90E+02, 2.44E+02 

64 

16 

2.71E+02, 4.78E+01 

1.19E+05, 4.19E+04 

64 

32 

1.71E+04, 3.02E+03 

4.91E+09, 1.74E+09 

128 

8 

5.85E+01, 1.03E+01 

1.00E+03, 3.53E+02 

128 

16 

4.03E+02, 7.13E+01 

1.77E+05, 6.24E+04 

128 

32 

2.70E+04, 4.77E+03 

7.77E+09, 2.75E+09 

256 

8 

8.38E+01, 1.48E+01 

1.43E+03, 5.06E+02 

256 

16 

5.85E+02, 1.03E+02 

2.56E+05, 9.05E+04 

256 

32 

4.02E+04, 7.11E+03 

1.16E+10, 4.09E+09 


Table |93] displays the condition numbers n(V s ) of the 3 q x 3 q Vandermonde Matrices defined by quasi- 
cyclic sequence of knots for q = 2 k and integers k. The table also displays the lower bounds, k and n ', on 
these numbers calculated based on equations (1531) and (15.31) . respectively. 
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Table 9.3: Condition Numbers of Vandermonde Matrices defined by quasi-cyclic sequence of knots 


n 

q 

k{V b ) 

K 

k' 

12 

4 

2.16e+01 

1.96e+01 

1.03E+01 

24 

8 

1.50e+03 

2 .22e+02 

1.06E+02 

48 

16 

1.16e+07 

2.01e+04 

1.13E+04 

96 

32 

9.86e+14 

1.16e+08 

1.27E+08 


Table [974] displays the condition numbers «(f l^) of the q x q leading blocks of the n x n DFT matrices 
Q = for n = 2q = 8,16,32,64. For the same pairs of n and q , the table also displays the lower- 

bounds, K- and k'_, on these numbers calculated based on Theorem 16.II and equation (15.31) for s(x) = p q (x), 
respectively. 

The test results show that the lower bounds based on equations (15.51) and (15.31) and even the ones based 
on Theorem 16.II grew very fast, but the condition numbers grew even faster. 


Table 9.4: Condition Numbers of the Leading Blocks of DFT matrices 


n 

q 




8 

4 

1.53E+01 

8.00E+00 

1.03E+01 

16 

8 

1.06E+03 

4.53E+01 

1.06E+02 

32 

16 

8.18E+06 

1.02E+03 

1.13E+04 

64 

32 

8.44E+14 

3.71E+05 

1.27E+08 


Table 19751 shows the mean and standard deviation of the relative residual norms rn = ||flfcX — b||/||b|| 
when GENP was applied to the linear systems of n equations fl„x = b, for the n x n DFT matrix f l n , 
vectors b of dimension n filled with i.i.d. standard Gaussian random variables, n = 64,128, 256, 512,1024, 
and 100 linear systems fl n x = b have been solved for each n. The tests have confirmed that the relative 
residual norms grew very fast for these inputs as n increased. 


Table 9.5: GENP with DFT Matrices: Relative Residual Norms 


n 

mean 

std 

16 

8.88E-14 

2.75E-14 

32 

8.01E-10 

3.56E-10 

64 

5.31E-03 

1.83E-03 

128 

5.00E+00 

2.15E+00 

256 

7.87E+02 

5.20E+02 

512 

7.05E+03 

4.44E+03 

1024 

6.90E+04 

3.26E+04 


10 Conclusions 

Our estimates for the condition numbers of n x n Vandermonde matrices 14 = (sj )i' J-o , having complex 
knots sq, , s n —\, provide some missing formal support for the well known empirical observation that such 
matrices tend to be badly ill-conditioned already for moderately large integers n. Our results also indicate 
that all the exceptions are narrowed to the Vandermonde matrices (such as the matrices of the discrete 
Fourier transform) whose knots are more or less equally spaced on or about the unit circle {x : \x = 11}. 
Our auxiliary inversion formulae for Cauchy and Vandermonde matrices may be of independent interest. 
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We began our study with readily deducing from the Ekkart-Young Theorem that the condition number 
k(V s ) is exponential in n unless all knots so,..., s ra _i lie in or near the unit disc {x : |a:| < 1} and furthermore 
unless they lie mostly on or near the unit circle (7(0,1). 

Then, by applying more involved techniques, based on the Vandermonde-Cauchy transformation of 
matrix structure, our formulae for the inversion of a Cauchy matrix, their extension to Vandermonde matrix 
inversion, and the techniques of low-rank approximation of Cauchy matrices, we prove our exponential lower 
bounds on the condition numbers of Vandermonde matrices 14, which hold unless the knots sq,..., s n _i are 
more or less equally spaced on or near the unit circle (7(0,1). 

Our test results are in a rather good accordance with our estimates. One can try to narrow the remaining 
gap between the estimated and observed values of the condition number k(V s ) by varying the parameters tj 
and / that we substitute into our basic equation (IQD . 

All our estimates hold for the transpose of a Vandermonde matrix as well. Furthermore we can extend 
our upper bounds on the smallest positive singular value a n (V s ) = 1/||V4|| to the upper bounds 

d d 

<Tn(V) < a n (\ 4) 53110,11 Ill'll and a n (C) < cr n ((7 Si t) El All HAII 

3 =1 3 =1 

on the smallest positive singular values a n {V) and <r„((7), respectively, of the “Vandermonde-like” and 
“Cauchy-like” matrices of the important classes V = JY =1 DjV s Cj and (7 = J^j=i D'jC s ^D ", respectively, 
for a bounded positive integer d , diagonal matrices Dj, D' r and D ", and factor-circulant (structured) matrices 
(7j, j = 1,... ,d (cf. .1*1)1 . Examples 1.4.1 and 4.4.6]). 

Most of our results can be readily extended to rectangular Vandermonde, Cauchy, “Vandermonde-like”, 
and “Cauchy-like” matrices, and a natural research challenge is their extension to polynomial Vandermonde 
matrices Vp, s = (pj(a;i))™E 0 ’" _1 where P = {pj{x))™Z q is any basis in the space of polynomials of degree 
less than n, for example, the basis made up of Cliebyshev polynomials. Equation (14.41) is extended to this 
case as follows (cf. EH] equation (3.6.8)]), 

( 1 \ Tn—l,n—l 

- -r) = diag(t(s i )“ 1 )^o 1 V PiS Vpj diag(t , (t i ))^E 0 1 . 

Here t(x) = nj=o C 2 " — A?) and t = (Ai)j=o denotes the coefficient vector of the polynomial t(x) — x n . The 
decisive step could be the selection of a proper set of knots T = { tj } jZ 0 , which would enable us to reduce 
the estimation of the condition number of a polynomial Vandermonde matrix Vp, s to the same task for a 
Cauchy matrix (7 s ,t- |G901 Sections 5 and 6] might give us some initial guidance for the selection of the knot 
set, that would extend the set of equally spaced knots on the unit circle, used in the case of Vandermonde 
matrices. Then it would remain to extend our estimates for CV matrices to the new subclass of Cauchy 
matrices. 

Acknowledgements: This work has been supported by NSF Grant CCF-1116736 and PSC CUNY 
Award 67699-00 45. 
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